programming concept
Synthesizing High-Quality Programming Tasks with LLM-based Expert and Student Agents
Nguyen, Manh Hung, Pădurean, Victor-Alexandru, Gotovos, Alkis, Tschiatschek, Sebastian, Singla, Adish
Generative AI is transforming computing education by enabling the automatic generation of personalized content and feedback. We investigate its capabilities in providing high-quality programming tasks to students. Despite promising advancements in task generation, a quality gap remains between AI-generated and expert-created tasks. The AI-generated tasks may not align with target programming concepts, could be incomprehensible to students, or may contain critical issues such as incorrect tests. Existing works often require interventions from human teachers for validation. We address these challenges by introducing PyTaskSyn, a novel synthesis technique that first generates a programming task and then decides whether it meets certain quality criteria to be given to students. The key idea is to break this process into multiple stages performed by expert and student agents simulated using both strong and weaker generative models. Through extensive evaluation, we show that PyTaskSyn significantly improves task quality compared to baseline techniques and showcases the importance of each specialized agent type in our validation pipeline. Additionally, we conducted user studies using our publicly available web application and show that PyTaskSyn can deliver high-quality programming tasks comparable to expert-designed ones while reducing workload and costs, and being more engaging than programming tasks that are available in online resources.
- Education > Educational Technology > Educational Software (0.68)
- Education > Curriculum > Subject-Specific Education (0.47)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Agents (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning > Generative AI (0.35)
Detecting Struggling Student Programmers using Proficiency Taxonomies
Schwartz, Noga, Fairstein, Roy, Segal, Avi, Gal, Kobi
Early detection of struggling student programmers is crucial for providing them with personalized support. While multiple AI-based approaches have been proposed for this problem, they do not explicitly reason about students' programming skills in the model. This study addresses this gap by developing in collaboration with educators a taxonomy of proficiencies that categorizes how students solve coding tasks and is embedded in the detection model. Our model, termed the Proficiency Taxonomy Model (PTM), simultaneously learns the student's coding skills based on their coding history and predicts whether they will struggle on a new task. We extensively evaluated the effectiveness of the PTM model on two separate datasets from introductory Java and Python courses for beginner programmers. Experimental results demonstrate that PTM outperforms state-of-the-art models in predicting struggling students. The paper showcases the potential of combining structured insights from teachers for early identification of those needing assistance in learning to code.
- North America > United States > North Carolina (0.04)
- Asia > Middle East > Israel (0.04)
- Instructional Material > Course Syllabus & Notes (1.00)
- Research Report > New Finding (0.88)
- Education > Focused Education > Special Education (0.85)
- Education > Educational Setting (0.68)
Web vs. LLMs: An Empirical Study of Learning Behaviors of CS2 Students
Kumar, Aayush, Prol, Daniel, Alipour, Amin, Ragavan, Sruti Srinivasa
LLMs such as ChatGPT have been widely adopted by students in higher education as tools for learning programming and related concepts. However, it remains unclear how effective students are and what strategies students use while learning with LLMs. Since the majority of students' experiences in online self-learning have come through using search engines such as Google, evaluating AI tools in this context can help us address these gaps. In this mixed methods research, we conducted an exploratory within-subjects study to understand how CS2 students learn programming concepts using both LLMs as well as traditional online methods such as educational websites and videos to examine how students approach learning within and across both scenarios. We discovered that students found it easier to learn a more difficult concept using traditional methods than using ChatGPT. We also found that students ask fewer follow-ups and use more keyword-based queries for search engines while their prompts to LLMs tend to explicitly ask for information.
- North America > United States > New York > New York County > New York City (0.05)
- North America > United States > Texas > Harris County > Houston (0.04)
- Europe > Spain (0.04)
- Asia > India > Karnataka > Bengaluru (0.04)
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (1.00)
- Instructional Material (1.00)
- Education > Curriculum > Subject-Specific Education (0.56)
- Education > Educational Setting > Higher Education (0.48)
Iris: An AI-Driven Virtual Tutor For Computer Science Education
Bassner, Patrick, Frankford, Eduard, Krusche, Stephan
Integrating AI-driven tools in higher education is an emerging area with transformative potential. This paper introduces Iris, a chat-based virtual tutor integrated into the interactive learning platform Artemis that offers personalized, context-aware assistance in large-scale educational settings. Iris supports computer science students by guiding them through programming exercises and is designed to act as a tutor in a didactically meaningful way. Its calibrated assistance avoids revealing complete solutions, offering subtle hints or counter-questions to foster independent problem-solving skills. For each question, it issues multiple prompts in a Chain-of-Thought to GPT-3.5-Turbo. The prompts include a tutor role description and examples of meaningful answers through few-shot learning. Iris employs contextual awareness by accessing the problem statement, student code, and automated feedback to provide tailored advice. An empirical evaluation shows that students perceive Iris as effective because it understands their questions, provides relevant support, and contributes to the learning process. While students consider Iris a valuable tool for programming exercises and homework, they also feel confident solving programming tasks in computer-based exams without Iris. The findings underscore students' appreciation for Iris' immediate and personalized support, though students predominantly view it as a complement to, rather than a replacement for, human tutors. Nevertheless, Iris creates a space for students to ask questions without being judged by others.
- Research Report > New Finding (1.00)
- Questionnaire & Opinion Survey (1.00)
- Instructional Material > Course Syllabus & Notes (0.93)
- Research Report > Experimental Study (0.68)
- Education > Educational Setting (1.00)
- Education > Curriculum > Subject-Specific Education (1.00)
- Education > Educational Technology > Educational Software > Computer Based Training (0.67)
- Information Technology > Artificial Intelligence > Representation & Reasoning (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Chatbot (0.93)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.90)
Do Large Code Models Understand Programming Concepts? A Black-box Approach
Hooda, Ashish, Christodorescu, Mihai, Allamanis, Miltos, Wilson, Aaron, Fawaz, Kassem, Jha, Somesh
Large Language Models' success on text generation has also made them better at code generation and coding tasks. While a lot of work has demonstrated their remarkable performance on tasks such as code completion and editing, it is still unclear as to why. We help bridge this gap by exploring to what degree auto-regressive models understand the logical constructs of the underlying programs. We propose Counterfactual Analysis for Programming Concept Predicates (CACP) as a counterfactual testing framework to evaluate whether Large Code Models understand programming concepts. With only black-box access to the model, we use CACP to evaluate ten popular Large Code Models for four different programming concepts. Our findings suggest that current models lack understanding of concepts such as data flow and control flow.
- North America > United States > Wisconsin > Dane County > Madison (0.04)
- Europe > Portugal > Lisbon > Lisbon (0.04)
- Europe > France (0.04)
Boldly Going Where No Benchmark Has Gone Before: Exposing Bias and Shortcomings in Code Generation Evaluation
Motivated by the increasing popularity of code generation from human descriptions using large language models (LLMs), several benchmarks have been proposed to assess the capabilities of existing and emerging models. This study presents a large-scale human evaluation of HumanEval and MBPP, two widely used benchmarks for Python code generation, focusing on their diversity and difficulty. Our findings reveal a significant bias towards a limited number of programming concepts, with negligible or no representation of most concepts. Additionally, we identify a concerningly high proportion of easy programming questions, potentially leading to an overestimation of model performance on code generation tasks.
- Information Technology > Software > Programming Languages (1.00)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Automatic Programming (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.68)
C++ Developer openings in San Francisco Bay Area, United States on August 01, 2022
Snap is proud to be an equal opportunity employer, and committed to providing employment opportunities regardless of race, religious creed, color, national origin, ancestry, physical disability, mental disability, medical condition, genetic information, marital status, sex, gender, gender identity, gender expression, pregnancy, childbirth and breastfeeding, age, sexual orientation, military or veteran status, or any other protected classification, in accordance with applicable federal, state, and local laws. If you have a disability or special need that requires accommodation, please don't be shy and contact us at_ _accommodations- • *About Us** Snap Inc. is a camera company. We believe that reinventing the camera represents our greatest opportunity to improve the way people live and communicate. Our products empower people to express themselves, live in the moment, learn about the world, and have fun together. At Snap, we believe that having a team of diverse backgrounds and voices working together will enable us to create innovative products that improve the way people live and communicate. Snap is proud to be an equal opportunity employer, and committed to providing employment opportunities regardless of race, religious creed, color, national origin, ancestry, physical disability, mental disability, medical condition, genetic information, marital status, sex, gender, gender identity, gender expression, pregnancy, childbirth and breastfeeding, age, sexual orientation, military or veteran status, or any other protected classification, in accordance with applicable federal, state, and local laws.
- North America > United States > California > San Francisco County > San Francisco (0.42)
- Pacific Ocean > North Pacific Ocean > San Francisco Bay (0.41)
- North America > United States > Colorado (0.04)
Python Programming - For Every Beginners
This course is aimed at offering the fundamental concepts of core Python programming language to the all levels of learners. This course is aimed at offering the fundamental concepts of core Python programming language to the all levels of learners. You will learn through videos, visual organizers and practice exercises. For a great hands-on learning experience, this course is packed with assignments, assessment tests, code challenges, quizzes, and exercises. Part-1: Core Python Programming Basics- starts with the basics of Python programming concepts like introduction, history & versions, features, uses, applications of python, data types, operators and control flow statements.
100%OFF
This course fully covers from classical C to Modern C style of creating object Oriented Programs from scratch to advance level in a step-by-step approach. The course teaches in detail the latest concepts introduced in C 11, C 14 and C 17. The object oriented programming concepts are covered in detail such that you will learn all the concepts including classes, objects, Data Abstraction, Data Encapsulation, Inheritance, polymorphism (including Operator overloading and Function Overloading). The main focus of the course apart from Fundamentals of programming and Object Oriented Programming is on Templates(including Function and Class Templates), which is a building block to understand STL implementation. The Course entirely covers all String Functions included in the latest version of C along with the basic programming concepts like operators, variables, Conditional statements and looping structures, functions(User-Defined and Recursive Functions), reference parameters, Arrays,File I/O and vectors in C .has been discussed in details.
The 7 Best Ways to Learn How to Code for Free
You've probably come across the term'coding' plenty of times, and if you haven't, then this is the best place to start. As we progress into the 21st century, the need for code continues to increase. Coding used to be limited to computers and video games, but now it encompasses every part of our lives. Coding is now an essential part of most major industries such as healthcare, finance, engineering, etc. Read on as we walk you through the basics of coding and how you, too, can learn to code. Coding, in essence, is the ability to make a computer do a particular task through instructions written in a programming language.
- Education > Educational Setting > Online (1.00)
- Education > Educational Technology > Educational Software > Computer Based Training (0.33)